Image processing apparatus, image capturing apparatus, and image processing method

ABSTRACT

An image processing apparatus comprises an acquisition unit configured to acquire a plurality of images captured under different exposure conditions, a detection unit configured to detect a position shift of an image to the reference image, wherein the reference image is an image serving as an alignment reference in processing for composing the plurality of images; and a composition unit configured to correct the position shift and compose the plurality of images in accordance with a composition ratio calculated based on brightness of a predetermined image, wherein the composition unit does not compose a plurality of images according to the composition ratio in a region of the reference image in which at least parts of the plurality of images do not overlap, and the composition unit composes images generated from the predetermined image.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a technique for controlling enlargementof a dynamic range in an image capturing apparatus.

2. Description of the Related Art

In general, to obtain one image, there is known dynamic rangeenlargement control for composing a plurality of images into one imageto generate an image having a wide dynamic range. This control aims atimproving an image quality by generating a composite image using imagesshot under different exposure conditions. Since shooting is performed aplurality of times, the shooting timings of the respective images aredifferent from each other. For example, when a moving object is shot,images may often be shot at different angles of view.

Japanese Patent Laid-Open No. 2008-109176 discloses a method ofoutputting a plurality of captured images by performing exposure aplurality of times and multiplying the output images by a gaincalculated by a ratio of the average luminance value of reference imagedata and a reference luminance value, thereby equalizing the brightnessof the reference image data and the brightness of non-reference imagedata to detect a position shift. Using the position shift detectionresult, the position shift of an image obtained before gainmultiplication is corrected, and image composition is performed.

However, in Japanese Patent Laid-Open No. 2008-109176, for example,assume that an image is rotated for alignment. Since one image does nothave peripheral pixels in image composition, as shown in FIG. 2, adesired image may not be obtained in the peripheral portion of thecomposite image.

SUMMARY OF THE INVENTION

The present invention has been made in consideration of the aboveproblem to make it possible to maintain the angle of view of a compositeimage in processing of aligning a plurality of images captured indifferent exposure conditions and composing the images.

According to the first aspect of the present invention, there isprovided an image processing apparatus comprising: an acquisition unitconfigured to acquire a plurality of images captured under differentexposure conditions; a detection unit configured to detect a positionshift of an image except a reference image in the plurality images withreference to the reference image, wherein the reference image is animage serving as an alignment reference in processing for composing theplurality of images; and a composition unit configured to correct theposition shift and compose the plurality of images in accordance with acomposition ratio calculated based on brightness of a predeterminedimage included in the plurality of images, wherein the composition unitdoes not compose a plurality of images according to the compositionratio in a region of the reference image in which at least parts of theplurality of images do not overlap, and the composition unit composesimages generated from the predetermined image.

According to the second aspect of the present invention, there isprovided an image processing method comprising: an acquisition step ofacquiring a plurality of images captured under different exposureconditions; a detection step of detecting a position shift of an imageexcept a reference image in the plurality images with reference to thereference image, wherein the reference image is an image serving as analignment reference in processing for composing the plurality of images;and a composition step of correcting the position shift and compose theplurality of images in accordance with a composition ratio calculatedbased on brightness of a predetermined image included in the pluralityof images, wherein in the composition step, a plurality of images arenot composed according to the composition ratio in a region of thereference image in which at least parts of the plurality of images donot overlap, and images generated from the predetermined image arecomposed.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a digital camera as the first embodiment ofan image capturing apparatus according to the present invention;

FIG. 2 is a view showing the concept of image alignment;

FIG. 3 is a flowchart showing the operation of the digital cameraaccording to the first embodiment of the present invention;

FIG. 4 is a view showing the relationship between luminance values andcomposition ratios;

FIG. 5 is a view showing an inscribed rectangle in image alignment; and

FIG. 6 is a flowchart showing the operation of a digital cameraaccording to the third embodiment of the present invention.

DESCRIPTION OF THE EMBODIMENTS

Embodiments of the present invention will be described in detail belowwith reference to the accompanying drawings.

First Embodiment

FIG. 1 is a block diagram of a digital camera 100 as the firstembodiment of an image capturing apparatus according to the presentinvention.

Referring to FIG. 1, the digital camera includes a shooting lens 10. Amechanical shutter 12 has a stop function. An image sensor 14 convertsan optical image into an electrical signal. An A/D converter 16 convertsan analog signal output from the image sensor 14 into a digital signal.

A timing generation circuit 18 supplies clock signals and controlsignals to the image sensor 14 and the A/D converter 16. The timinggeneration circuit 18 is controlled by a memory control circuit 22 and asystem control circuit 50. In addition to the mechanical shutter 12, thedigital camera includes an electronic shutter to control theaccumulation time by controlling the reset timings of the image sensor14. The digital camera can be used for capturing a moving image.

An image processing circuit 20 performs predetermined pixelinterpolation processing and color conversion processing for data fromthe A/D converter 16 or data from the memory control circuit 22. Theimage processing circuit 20 extracts an image and performs zooming,thereby implementing an electronic zoom function.

The image processing circuit 20 performs predetermined arithmeticprocessing using the captured image data. The system control circuit 50performs AF processing, AE processing, and EF processing for controllingan exposure control unit 40 and a distance measurement control unit 42using the resultant arithmetic result. The image processing circuit 20performs predetermined arithmetic processing using the captured imagedata and also performs AWB (Auto White Balance) processing based on theresultant arithmetic result.

The memory control circuit 22 controls the A/D converter 16, the timinggeneration circuit 18, the image processing circuit 20, a memory 30, anda compression/decompression circuit 32. The data from the A/D converter16 is written in the memory 30 via the image processing circuit 20 andthe memory control circuit 22. Alternatively, the data from the A/Dconverter 16 is written in the memory 30 via only the memory controlcircuit 22. An image display unit 28 includes an TFT LCD or the like.The display image data written in the memory 30 is displayed on theimage display unit 28 via the memory control circuit 22.

When the captured image data are sequentially displayed using the imagedisplay unit 28, the electronic viewfinder function can be implemented.The image display unit 28 can arbitrarily change the display between ONand OFF according to an instruction from the system control circuit 50.When the display is set OFF, the power consumption of the digital camera100 can be greatly reduced.

The memory 30 stores the captured still images and moving images. Thememory 30 has a storage capacity enough to store a predetermined numberof still images and a moving image of a predetermined time. Images canbe written in the memory 30 at high speed in a large volume in panoramicshooting and continuous shooting for continuously shooting a pluralityof still images. The memory 30 can also be used as a work area for thesystem control circuit 50.

The compression/decompression circuit 32 compresses and decompressesimage data using ADCT (Adaptive Discrete Cosine Transform). Thecompression/decompression circuit 32 receives an image stored in thememory 30 and performs compression processing or decompressionprocessing of the image. The processed data is written in the memory 30.

The exposure control unit 40 controls the shutter 12 having the stopfunction. The exposure control unit 40 also has flash light controlfunction in synchronism with an electronic flash 48. The distancemeasurement control unit 42 controls focusing of the shooting lens 10. Azoom control unit 44 controls zooming of the shooting lens 10.

The electronic flash 48 has an AF auxiliary light projection functionand the flash light control function. The exposure control unit 40 andthe distance measurement control unit 42 are controlled using the TTLmethod. The system control circuit 50 controls the exposure control unit40 and the distance measurement control unit 42 based on the arithmeticresult of the captured image data from the image processing circuit 20.The system control circuit 50 controls the entire system of the digitalcamera 100.

Operation members 60, 62, 64, 66, 70, and 72 are used to input variousoperation instructions to the system control circuit 50. These operationmembers are formed from switches, a dial, a touch panel, a pointingelement using line-of-sight detection, a speech recognition device, andthe like singly or in a combination. A detailed explanation of theseoperation members will be made below.

The mode dial switch 60 switches a variety of functional modes such aspower off, automatic shooting mode, shooting mode, panoramic shootingmode, moving image capturing mode, playback mode, and PC connectionmode.

The shutter switch SW1 62 is half depressed (half stroke) and turned onand instructs the start of operation such as AF (Auto Focus) processing,AE (Auto Exposure) processing, and AWB (Auto White Balance) processing.The shutter switch SW2 64 is fully depressed (full stroke) and turnedon. In electronic flash shooting, EF (Electronic Flash pre-emission) isperformed, and the image sensor 14 is exposed for an exposure timedetermined by AE processing. The electronic flash emits light duringthis exposure period, and the exposure control unit 40 controls to turnoff the electronic flash at the end of the exposure period, therebycompleting the exposure to the image sensor 14. After that, readoutprocessing for writing, as image data in the memory 30 via the A/Dconverter 16 and the memory control circuit 22, the signal read out fromthe image sensor 14, and development processing using arithmeticoperations in the image processing circuit 20 and the memory controlcircuit 22 are performed. In addition, recording processing for readingout image data from the memory 30, compressing it by thecompression/decompression circuit 32, and writing the image data on arecording medium 200 is performed. The shutter switch SW2 instructs thestart of a series of shooting processing operations.

The display selection switch 66 can perform display selection of theimage display unit 28. When shooting is performed using an opticalviewfinder 104 using the above function, a current is cut off from theimage display unit made of a TFT LCD or the like, thereby achievingpower saving.

The operation unit 70 includes various buttons, a touch panel, and arotary dial, and namely a menu button, set button, macro button,multi-screen playback repagination button, electronic flash settingbutton, single shooting/continuous shooting/self-timer selection button,and the like. The operation unit 70 also includes a menu move + (plus)button, menu move − (minus) button, playback image move + (plus) button,playback image move − (minus) button, shooting image quality selectionbutton, exposure correction button, date/time setting button, and thelike.

The zoom switch 72 serves as a zoom operation unit for allowing the userto instruct the zoom ratio of a captured image. The zoom switch 72includes a tele switch for changing the shooting angle of view towardthe telephoto side and a wide switch for changing the shooting angle ofview toward the wide-angle side. The use of this zoom switch 72 causesthe zoom control unit 44 to trigger the instruction for changing theshooting angle of view of the shooting lens 10 and the optical zoomoperation. The use of the zoom switch 72 also triggers image extractionby the image processing circuit 20 and the change in electronic zoomingin shooting angle of view using pixel interpolation processing or thelike.

A power supply unit 86 is made of a primary battery such as an alkalibattery, a secondary battery such as an NiCd battery, NiMH, battery, orLi ion battery, or an AC adapter. An interface 90 serves as an interfacewith a memory card, hard disk, or the like. A connector 92 connects to arecording medium such as a memory card or hard disk.

The optical viewfinder 104 can perform shooting singly without using theelectronic viewfinder function of the image display unit 28. Acommunication unit 110 has various communication functions such as USB,IEEE1394, LAN, and wireless communication. Reference numeral 112 denotesa connector for connecting the digital camera 100 to another device, oran antenna for wireless communication, using the communication unit 110.

The recording medium 200 is made of a memory card, hard disk, or thelike. The recording medium 200 includes a recording unit 202 made of asemiconductor memory, magnetic disk, or the like, an interface 204 withthe digital camera 100, and a connector 206 connected to the digitalcamera 100.

FIG. 3 is a flowchart showing the operation of the digital cameraaccording to this embodiment.

In step S301, the user shoots three images under different exposureconditions, that is, an under image (underexposure image), a correctimage (correct exposure image), and an over image (overexposure image).In step S302, different gains are multiplied to the under image, correctimage, and over image obtained in step S301. At this time, optimal gainsare multiplied to the images, respectively. That is, gains are set suchthat the respective images have brightness levels, respectively,corresponding to the correct exposure.

In step S303, a composite gain obtained by composing the gains of therespective images as indicated by equation (1) is multiplied to oneunder image obtained in step S301. A composite gain Gg(Y) is given by:

Gg(Y)=α*Gl(Y)+β*Gm(Y)+γ*Gh(Y)  (1)

for α+β+γ=1, 0≦α≦1, 0≦α≦1, 0≦β≦1, 0≦γ≦1

where Y is the pixel luminance value of the under image, Gl(Y), Gm(Y),and Gh(Y) are the under image gain, correct image gain, and over imagegain, respectively, α, β, and γ are parameters representing thecomposition ratios of the under image, correct image, and over image.The composition ratios of the gains of the respective images areadjusted such that the under image has a brightness level correspondingto the correct exposure by multiplying the composite gain, therebydetermining the gains. In this case, the composition ratio is adjusted,and a gain equal to the gain used in the under image in step S302 isapplied.

In step S304, the images (first under image, correct image, and overimage) obtained in step S302 and the image (second under image) obtainedin step S303 are developed. In this embodiment, “first underimage=second under image” may be set, and development may be done atonce.

In step S305, the first under image, correct image, and over imageobtained in step S304 are composed. The three images to be composed aregiven as aligned images. For example, image rotation or the like isperformed to align the correct image on the first under image, align theover image on the first under image, and compose the aligned correct andover images with the first under image. More specifically, out of theplurality of images, the position shifts of images except the referenceimage are detected with reference to the reference image, and the imagesfree from the position shifts are composed to obtain a composite image.At this time, the composition ratio of the image is determined, forexample, based on a luminance value Yl of each pixel of the first underimage, as shown in FIG. 4, using equation (2), that is, a pixel value Gof the composite image:

G=α2*L+β2*M+γ2*H  (2)

for

-   -   α2=0, β2=0, γ2=1 if 0≦Yl≦a    -   α2=0, β2=(Yl−a)/((b−a), γ2=1−β2 if a≦Yl≦b    -   β2=0, β2=1, γ2=0 if b≦Yl≦c    -   α2=(Yl−c)/(−c), β2=1−α2, γ2=0, if c≦Yl≦d    -   α2=1, β2=0, γ2=0 if d≦Yl        where L, M, and H are the pixel values of the under image,        correct image, and over image, respectively, and α2, β2, and γ2        are parameters representing the composition ratios of the under        image, correct image, and over image, respectively. More        specifically, a low-luminance pixel is applied from the over        image, a medium-luminance pixel is applied from the correct        image, and a high-luminance pixel is applied from the under        image. The levels between them are made linear. When differences        between the angles of view of the images are large and alignment        fails, the first under image is made as the final output (step        S306).

In step S307, the second under image obtained in step S304 and theoutput image in step S305 are composed. In this embodiment, to simplifythe hardware configuration and reduce the processing cost, as shown inFIG. 5, the coordinates of an inscribed rectangle (rectangle region)with respect to two images to be composed are calculated, and outerpixels of the inscribed rectangle are replaced with the second underimage. The resultant images are then composed. That is, the referenceimage is used regardless of image brightness in a predetermined regionincluding a region in which parts of the plurality of aligned images donot overlap in the reference image. The inscribed rectangle serves as analignment target and is a region in an output image defined byintersections obtained by extending lines parallel or perpendicular tothe sides of the reference image from the respective vertices of ageometrically deformed image. The image processing circuit 20 accordingto this embodiment has a hardware configuration which calculates thecoordinates of the above inscribed rectangle and performs processesinside and outside the rectangle separately. Alternatively, thecomposition ratios of the two images are linearly changed with apredetermined width for the boundary portion defining the inside andoutside of the inscribed rectangle, thereby making the boundaryunnoticeable.

As described above, according to this embodiment, in processing foraligning and composing a plurality of images captured under differentexposure conditions, the reference image is used as an image in a regionin which at least one image does not exist except the image (underimage) serving as the alignment reference in the composite image. Thismakes it possible to maintain, in the composite image, the angles ofview of the images before the composition.

Second Embodiment

The second embodiment will now be described below. The arrangement of adigital camera according to the second embodiment as an example of theimage capturing apparatus of the present invention is the same as thearrangement of the digital camera 100 according to the first embodimentshown in FIG. 1. A description of the digital camera of the secondembodiment will be omitted.

The operation of the digital camera of this embodiment is the same asthat of the first embodiment shown in FIG. 3 except steps S302 and S303.The rest of processing is the same as that of the first embodiment, andthe explanation of the common part will be omitted.

In step S302, each of an under image, correct image, and over imageobtained in step S301 is divided into M×N blocks. In each block, thesignals values of the respective pixels are added and averaged to obtaina signal value of each block. A gain is multiplied to each image basedon the signal value of each block. At this time, the gains are set suchthat the output pixel values (output values) of the gained imagesincrease in the order of the under image, correct image, and over image.With this arrangement, for example, when an object and a background arepresent in a composite image in step S305, an effect can be producedsuch that the background around the object may be made bright as ifbacklight were emitted to the object or an image has an atmosphere as ifa shadow were formed.

In step S303, a composite gain generated by composing the gains of therespective images as indicated by equation (1) is multiplied to theunder image obtained in step S301. At this time, the gain is composedbased on the composition ratio of the image as indicated by equation (2)used for composition in step S305. The composition ratio of the imagehere is the composition ratio for the developed image. For this reason,the composition ratio can be used by converting it into the compositionratio of the image before the development. With this operation, duringimage composition in step S307, the brightness step between the insideof the image and the peripheral portion of the image can be madeunnoticeable, thereby providing a desirable image to the user.

As described above, according to this embodiment, in processing foraligning and composing a plurality of images captured under differentexposure conditions, the reference image is used as an image in a regionin which at least one image does not exist except the image serving asthe alignment reference in the composite image. This makes it possibleto maintain, in the composite image, the angles of view of the imagesbefore the composition.

Third Embodiment

The third embodiment will now be described below. The arrangement of adigital camera according to the third embodiment is the same as thearrangement of the digital camera 100 according to the first embodimentshown in FIG. 1. A description thereof will be omitted.

FIG. 6 is a flowchart showing the operation of the digital cameraaccording to this embodiment.

In step S601, the user shoots three images under different exposureconditions, that is, an under image (underexposure image), a correctimage (correct exposure image), and an over image (overexposure image).

In step S602, the under image, correct image, and cover image obtainedin step S601 are divided into M1×N1 blocks. The signal values of thepixels of each block are added and averaged to obtain a signal value ofeach block.

A gain is multiplied to each image based on the signal value of eachblock in step S603. At this time, the gains are set such that the outputpixel values (output values) of the gained images increase in the orderof the under image, correct image, and over image. With thisarrangement, for example, when an object and a background are present ina composite image in step S607, an effect can be produced such that thebackground around the object may be made bright as if backlight wereemitted to the object or an image has an atmosphere as if a shadow wereformed.

In step S604, the under image obtained in step S601 is divided intoM2×N2 (M2>M1 and N2>N1) blocks. The signals values of the pixels of eachblock are added and averaged to calculate a signal value of each block.The division number is increased to improve precision of the signalvalue of each bock and prevent occurrence of the luminance step in imagecomposition in step S609.

In step S605, a composite gain obtained by composing the gains of therespective images as indicated by equation (3) is multiplied to oneunder image obtained in step S601. A composite gain Gg(Y) is given by:

Gg(Y)=α*Gl(Y)+β*Gm(Y)+γ*Gh(Y)  (3)

for α+β+γ=1, 0≦α≦1, 0≦β≦1, 0≦γ≦1

where Y is the pixel luminance value of the under image, Gl(Y), Gm(Y),and Gh(Y) are the under image gain, correct image gain, and over imagegain, respectively, α, β, and γ are parameters representing thecomposition ratios of the under image, correct image, and over image,respectively. At this time, gains are composed based on the compositionratios of the images shown in FIG. 4 which are used for composition instep S607. Since the composition ratios of the images are thecomposition ratios of the developed images, the composition ratios areconverted into the composition ratios of the images before thedevelopment, thereby determining the gain.

In step S606, the images (first under image, correct image, and overimage) obtained in step S603 and the image (second under image) obtainedin step S605 are developed.

In step S607, the first under image, correct image, and over imageobtained in step S606 are composed. The three images to be composed aregiven as aligned images. For example, image rotation or the like isperformed to align the correct image on the first under image, align theover image on the first under image, and compose the aligned correct andover images with the first under image. More specifically, out of theplurality of images, the position shifts of images except the referenceimage are detected with reference to the reference image, and the imagesfree from the position shifts are composed to obtain a composite image.At this time, the composition ratio of the image is determined, forexample, based on a luminance value Yl of each pixel of the first underimage, as shown in FIG. 4, using equation (4), that is, a pixel value Gof the composite image:

G=β2*L+β2*M+γ2 *H  (4)

for

-   -   α2=0, β2=0, γ2=1 if 0≦Yl≦a    -   α2=0, β2=(Yl−a)/(b−a, γ2=1=β1 if a≦Yl≦b    -   α2=0, β2=1, γ2=0 if b≦Yl≦c    -   α2=(Yl−c)/(d−c), β2=1−α2, γ2=0 if c≦Yl≦c    -   α2=1, β2=0, γ2=0 if d≦Yl        where L, M, and H are the pixel values of the under image,        correct image, and over image, respectively, and α2, β2, and γ2        are parameters representing the composition ratios of the under        image, correct image, and over image, respectively. More        specifically, a low-luminance pixel is applied from the over        image, a medium-luminance pixel is applied from the correct        image, and a high-luminance pixel is applied from the first        under image. The levels between them are made linear. When        differences between the angles of view of the images are large        and alignment fails, the first under image is made as the final        output (step S608).

In step S609, the second under image obtained in step S606 and theoutput image obtained in step S607 are composed. When alignment isperformed in step S607, in this embodiment, to simplify the hardwareconfiguration and reduce the processing cost, as shown in FIG. 5, thecoordinates of an inscribed rectangle with respect to two images to becomposed are calculated, and outer pixels of the inscribed rectangle inthe output image obtained in step S606 are replaced with the secondunder image obtained in step S606. The resultant images are thencomposed. That is, the reference image is used regardless of imagebrightness in a predetermined region including a region in which partsof the plurality of aligned images do not overlap in the referenceimage. The inscribed rectangle serves as an alignment target and is aregion in an output image defined by intersections obtained by extendinglines parallel or perpendicular to the sides of the reference image fromthe respective vertices of a geometrically deformed image. The imageprocessing circuit 20 has a hardware configuration which calculates thecoordinates of the above inscribed rectangle and performs processesinside and outside the rectangle separately. Alternatively, thecomposition ratios of the two images are linearly changed with apredetermined width for the boundary portion defining the inside andoutside of the inscribed rectangle, thereby making the boundaryunnoticeable. With this arrangement, it is possible to make theluminance step between the inside of the image and the peripheralportion of the image unnoticeable when performing image composition instep S609, thereby providing a desirable image to the user.

As described above, according to this embodiment, in processing foraligning and composing a plurality of images captured under differentexposure conditions, the reference image is used as an image in a regionin which at least one image does not exist except the image serving asthe alignment reference in the composite image. This makes it possibleto maintain, in the composite image, the angles of view of the imagesbefore the composition.

Other Embodiments

Aspects of the present invention can also be realized by a computer of asystem or apparatus (or devices such as a CPU or MPU) that reads out andexecutes a program recorded on a memory device to perform the functionsof the above-described embodiment(s), and by a method, the steps ofwhich are performed by a computer of a system or apparatus by, forexample, reading out and executing a program recorded on a memory deviceto perform the functions of the above-described embodiment(s). For thispurpose, the program is provided to the computer for example via anetwork or from a recording medium of various types serving as thememory device (e.g., computer-readable medium).

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application No.2013-170828 filed Aug. 20, 2013 and Japanese Patent Application No.2013-170829 filed Aug. 20, 2013 which are hereby incorporated byreference herein in their entirety.

What is claimed is:
 1. An image processing apparatus comprising: anacquisition unit configured to acquire a plurality of images capturedunder different exposure conditions; a detection unit configured todetect a position shift of an image except a reference image in theplurality images with reference to the reference image, wherein thereference image is an image serving as an alignment reference inprocessing for composing the plurality of images; and a composition unitconfigured to correct the position shift and compose the plurality ofimages in accordance with a composition ratio calculated based onbrightness of a predetermined image included in the plurality of images,wherein said composition unit does not compose a plurality of imagesaccording to the composition ratio in a region of the reference image inwhich at least parts of the plurality of images do not overlap, and saidcomposition unit composes images generated from the predetermined image.2. The apparatus according to claim 1, wherein said composition unitcomposes images generated from the predetermined image in a region ofthe reference image outside a rectangular region in which a plurality ofimages having undergone position shift correction overlap.
 3. Theapparatus according to claim 1, further comprising a setting unitconfigured to set gains based on brightness levels of the plurality ofimages, wherein said composition unit composes the plurality of imagesto which the gains are applied, and composes, in the region in which atleast parts of the plurality of images do not overlap, images generatedby applying, to the predetermined image, a composite gain obtained bycomposing gains in accordance with the composition ratios.
 4. Theapparatus according to claim 3, wherein said setting unit divides eachof the plurality of images into blocks, and sets the gain based on asignal value calculated by adding and averaging signal values of eachblock.
 5. The apparatus according to claim 3, wherein said setting unitsets the gain in descending order of exposure levels of the plurality ofimages, which are represented by output values obtained uponrespectively applying the gains to the plurality of images.
 6. Theapparatus according to claim 1, wherein the predetermined image is animage having a lowest exposure level in the plurality of images.
 7. Theapparatus according to claim 1, wherein said composition unit dividesthe predetermined image into M2×N2 (M2>M1 and N2>N1) and calculates thecomposition ratio based on the signal value calculated by adding andaveraging the signal values of each block, and said setting unit divideseach of the plurality of images into M1×N1 blocks and sets the gainbased on a signal value calculated by adding and averaging signalsvalues of each block.
 8. An image capturing apparatus comprising: animage capturing unit configured to capture images under differentexposure conditions; and an image processing apparatus defined inclaim
 1. 9. An image processing method comprising: an acquisition stepof acquiring a plurality of images captured under different exposureconditions; a detection step of detecting a position shift of an imageexcept a reference image in the plurality images with reference to thereference image, wherein the reference image is an image serving as analignment reference in processing for composing the plurality of images;and a composition step of correcting the position shift and compose theplurality of images in accordance with a composition ratio calculatedbased on brightness of a predetermined image included in the pluralityof images, wherein in the composition step, a plurality of images arenot composed according to the composition ratio in a region of thereference image in which at least parts of the plurality of images donot overlap, and images generated from the predetermined image arecomposed.